CSE202 Project - Consistent Hashing

Consistent hashing is a special kind of hashing which is useful for certain applications. Originally devised by Karger et. al. at MIT[2] for use in distributed caching, the idea has now been expanded to other areas, most notably peer to peer networking[5][4]. As internet use increases and as distributed systems grow more prevalent (ie, Napster clones), consistent hashing could play an increasingly important role. At a basic level, consistent hashing is just a special kind of hashing. Namely, a consistent hash function is one which changes minimally as the range of the function changes[2]. This makes consistent hashing ideal in circumstances where the set of buckets changes over time. Furthermore, it can be shown that, with high probability, two users with inconsistent but overlapping sets of buckets will map items to the same buckets. This is particularly important because it eliminates the need to maintain consistent state among all nodes in a network. In this paper I present a family of consistent hash functions, attempt to prove certain properties of these functions, and describe an application of consistent hashing, namely peer to peer networking.